Systems and methods for monitoring agricultural products

ABSTRACT

The present invention relates to systems and methods for monitoring agricultural products. In particular, the present invention relates to monitoring fruit production, plant growth, and plant vitality. According to embodiments of the invention, a plant analysis system is configured determine a spectral signature of a plant based on spectral data, and plant color based on photographic data. The spectral signatures and plant color are associated with assembled point cloud data. Morphological data of the plant can be generated based on the assembled point cloud data. A record of the plant can be created that associates the plant with the spectral signature, plant color, spectral data, assembled point cloud data, and morphological data, and stored in a library.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of, and claims thebenefit of, of U.S. patent application Ser. No. 15/977,771 filed on May11, 2018, which is a continuation-in-part application of, and claims thebenefit of, and priority to, U.S. patent application Ser. No. 15/868,840filed on Jan. 11, 2018 (issued as U.S. Pat. No. 10,371,683), whichclaims priority to U.S. patent application Ser. No. 13/907,147 filed onMay 31, 2013 (issued as U.S. Pat. No. 9,939,417), which claims priorityto U.S. Provisional Patent Application No. 61/654,312, filed Jun. 1,2012, each of which are hereby incorporated by reference in theirentirety as if recited in full herein.

FIELD OF THE INVENTION

The present invention relates to systems and methods for monitoringagricultural products. In particular, the present invention relates toagricultural asset management, including: monitoring fruit production,tracking plant growth, monitoring plant vitality and productivity,measuring morphological attributes and performing field-basedphenotyping.

BACKGROUND OF THE INVENTION

Accurate and timely machine counting of fruit on the tree or vine haslong been considered impossible or impractical. Current methods rely onmanual estimation and are often inaccurate and labor intensive.Inaccurate estimates lead to inaccurate crop forecasts. This inaccuracycomplicates pricing and grower's ability to forecast plan, and optimizemarket timing participation. What is needed is an improved method foraccurately determining and forecasting plant size and quality andharvested yield.

Further, agricultural systems typically do not take into account how thegenetics of a plant can impact properties such as size and quality basedon observed phenotypical data.

SUMMARY OF THE INVENTION

The present invention relates to systems and methods for monitoringagricultural products. In particular, the present invention relates tomonitoring fruit production, plant growth, and plant vitality.

Embodiments of the present disclosure provide systems and methods forimproved fruit tree analysis and crop predictions. The systems andmethods described herein improve on the accuracy and efficiency ofexisting methods. The systems and methods of embodiments of the presentinvention find use in research, commercial agriculture, and forestry,among other uses.

For example, in some embodiments, the present invention provides ananalysis system, comprising: a) a data acquisition component; b)optionally, a transport component configured to transport the dataacquisition component to collect data on fruit trees or vines; and c) asoftware component configured to analyze the data to generate analyzeddata. In some embodiments, the transport component is a transportvehicle, The transport vehicle may be a four-wheel independentsuspension engine for transporting one or more sensors over an uneventerrain with minimal noise. The transport vehicle may have a cargo boxfor storing and supporting modular sensor mounts and fixtures. The cargobox may be formed with a high-density polyethylene copolymer compositematerial that eliminates rust, dents, and reduces noise.

In some embodiments, the transport vehicle is autonomous or remotelycontrolled. The transport vehicle can be autonomously controlled basedon assembled point cloud data to navigate around and through one or moreplants. For example, assembled point cloud data of an orchard, includingthe precise position of each plant in the orchard, can be stored on thetransport vehicle. The transport vehicle can then navigate through theorchard, and based on its current position as determined by the globalnavigation satellite system, and the location of that position relativeto the assembled point cloud, the transport vehicle can detect plants inits proximity and/or path and avoid colliding with the plants.

In some embodiments, the data acquisition component comprises one ormore devices selected from, for example, one or more of a 3D laserscanner, a survey grade GPS, thermal imaging, radio, sound and magneticwaves, a thermal imaging camera, multispectral and/or hyperspectralsensors, soil conductivity sensors or a high-speed, high-density (HD)video camera. In some embodiments, the data may include, for example,one or more of tree trunk diameter, height of tree, volume of tree, leafdensity of tree, color of leaves on tree, GPS location of tree, bar codedata for tree, number of blossoms on tree, presence of pests includingdiseases on the fruit or tree, subspecies of the tree, or an annotatedor un-annotated photograph of the tree. In some embodiments a fullthree-dimensional virtual representation of the tree is produced atsub-centimeter-level resolution.

The present invention is not limited to the analysis of a particularfruit tree or vine. Plants that can be analyzed by embodiments of theinvention can include without limitation permanent crop plants, croptrees, forestry, and corn. Examples include but are not limited to,abiu, acerola, almond, amla (indian gooseberry), apple, apricot, aprium,avocados, bael, bananas, ber (indian plum), blackberries, blood orange,blueberries, breadfruit, calamondin, cantaloupe melon, carambola(starfruit), cashew, the fruit, cherries, chestnut, chocolate,chokecherry, citron, cocoa, coconuts, coffee, corn plant, crabapple,cumaquat, currant, custard-apple, dates, dewberries, dragon fruit,durian, feijoa, fig, grapefruit, grapes, guava, hazelnut, honeydew,hops, jaboticaba, jackfruit, jujube, kaffir lime, key lime, kiwifruit,kumquat, lemons, limes, loganberries, longan, loquat, lychee, macadamia,mandarin, mangoes, mangosteen, medlar, morello cherry, mulberries, natalplum, nectarines, oil palm, olives, oranges, papayas, passion fruit,pawpaw, peaches, pears, pecan, persimmon, pineapples, plums, pluot,pomegranate, pomelo, pongamia, prune, pummel, pumpkin, raspberries, redbanana, rock melon, sabine, sapodilla (chikoo), sapote, soursop,starfruit, stone fruit, strawberries, strawberry tree, sugar-apple(sharifa), surinam cherry, tamarillo, tamarind, tangelos, tangerines,tomatoes, ugli, uglifruit/uniqfruit, walnut, watermelon, a grape vine, atomato vine, or an apple tree.

In some embodiments, the software component further comprises a computerprocessor and a user interface configured to provide or display theanalyzed data (e.g., to a user). In some embodiments, the analyzed datamay represent, for example, one or more of tree health, predicted fruityield, predicted fruit ripening period, or other features of interest.

In further embodiments, the present invention provides a method,comprising: a) collecting data on plants (e.g., fruit trees) using adata acquisition component transported by a transport component; and b)analyzing the data with a software component to generate analyzed data.In some embodiments, the method further comprises the step of using theanalyzed data to calculate spray dispersal and/or in order to guidefruit tree sprayers (e.g., to determine when to spray, how long tospray, and what chemicals to spray). In some embodiments, the methodfurther comprises the step of identifying species and/or subspecies ofthe tree. In some embodiments, the method further comprises the step ofidentifying disease in the tree and/or fruit.

In some embodiments, the invention may comprise a plant analysis systemthat includes a 3D laser scanner, a camera, a spectral sensor, and asurvey grade global positioning satellite (GPS) receiver. The surveygrade GPS receiver can be configured to measure location with a subcentimeter level of accuracy. The 3D laser scanner can be configured tomeasure properties of a plant utilizing light detection and ranging(LiDAR), including waveform LiDAR and assemble point cloud data.Assembled point cloud data can include three dimensional vertices, eachof which represents the external surface of an object on the plant. Thecamera can collect photographic data of the plant, the spectral sensorcan gather spectral data of the plant, and the GPS receiver cangeo-register the point cloud data and the photographic data. In someaspects of the invention, the system can fuse the data collected fromthe plant analysis system to enable the generation of morphological andphenotypical data for further analysis. Each three dimensional vertex ofthe assembled point cloud data can be associated with GPS data,photographic data, and spectral data.

The plant analysis system can include a transport vehicle that cantransport the 3D laser scanner, camera, spectral sensor, and surveygrade GPS receiver to collect the data on the plant. The transportvehicle can be a manned or unmanned, ground-based or airborne vehiclefor transporting the plant analysis system. In some embodiments, thespectral sensor can be an active hyperspectral sensor. The plantanalysis system can include a computer processor configured to determinea spectral signature of the plant based on the spectral data. Thecomputer processor can also be configured to determine plant color basedon photographic data and associate the assembled point cloud data withthe plant color. The computer processor can further generatemorphological data of the plant based on the assembled point cloud data,the morphological data comprising plant stem diameter, plant height,plant volume, and plant leaf density. The computer processor can createa record of the plant in a library. The record of the plant associatesthe plant with the spectral signature, the plant color, the spectraldata, the assembled point cloud data, and the morphological data.

In some embodiments, the computer processor of the plant analysis systemcan generate morphological data by segmenting the assembled point clouddata to identify boundaries of the plant. The computer processor canclassify the morphological data to identify a plant feature. The plantfeature can include a branching structure, trunk, biomass, canopy,fruit, blossom, fruit cluster, or blossom cluster.

In some embodiments, the computer processor of the plant analysis systemcan utilize the assembled point cloud data and the plant color todiscriminate a fruit, blossom, fruit cluster, or blossom cluster from ashadow. The discrimination can be based on analyzing a pixel-by-pixelvariation of the plant color and a geometric shape defined by thevertices of the assembled point cloud data.

In some embodiments, the plant analysis system can further includeatmospheric sensors for measuring atmospheric conditions. The computerprocessor can determine a phenotype of a plant based on the spectralsignatures, morphological data, and atmospheric conditions measured bythe atmospheric sensors.

In some embodiments, the computer processor of the plant analysis systemcan be configured to determine a vegetation index of one or more plantsbased on the spectral data. In yet other embodiments, the computerprocessor of the plant analysis system can be configured to determine anumber and a size of fruits or blossoms on a plant. The number and sizeof fruits or blossoms on a plant can be based on a fusion of theassembled point cloud data and plant color data. The number and size offruits or blossoms on a plant can be determined by clustering theassembled point cloud data and plant color data. The number and size offruits or blossoms on a plant can be used to calculate a crop yieldestimate. In some embodiments, additional data, such as spectral data,can be fused to enhance the counting and sizing of fruits or blossoms.

In some embodiments, the computer processor of the plant analysis systemcan be configured to compare a spectral signature of a plant with aspectral signature from the library of plant records based on thespectral information divergence of the spectral signatures. The computerprocessor can be further configured to detect the presence of a plantdisease based on the comparison of spectral signatures and imageanalysis. The computer processor can also be configured to detect thepresence of wilt or leaf drop caused by environmental stressors based onthe comparison of spectral signatures, point clouds, and image analysis.The computer processor can also be configured to identify a diseasevector and predict the disease vector's trajectory. In some embodiments,the library of plant records is configured to store historical dataassociated with a plant. The historical data can include informationsuch as the date of planting, root stock, grafting record, nutritionaltreatment, health treatment, yield, surrounding soil conditions,surrounding atmospheric conditions, and surrounding topography. In someembodiments, the plant analysis system can further include an assetmanagement dashboard for accessing, viewing, analyzing and controllingthe library of plant records.

Additional embodiments are described herein.

DESCRIPTION OF THE FIGURES

FIG. 1 shows an image of annotations of trees generated by systems andmethods of embodiments of the present invention.

FIG. 2 shows a task-methodology breakdown of systems and methods ofembodiments of the present invention.

FIG. 3 shows a schematic of an exemplary crop analysis system for fruitbearing trees and vine crops.

FIG. 4 shows a schematic of an exemplary crop analysis system for fruitbearing trees and vine crops.

FIG. 5 shows an example of a point cloud map of a fruit tree.

FIGS. 6a, 6b, and 6c show an asset management dashboard according tosome embodiments of the invention.

FIGS. 7a, 7b, and 7c show exemplary user interfaces for controlling adata acquisition component, a transport component, and sprayer accordingto some embodiments of the invention.

FIGS. 8a and 8b show examples of a data set being sized and countedbased on clustering algorithms according to some embodiments of theinvention.

FIG. 9 shows a system comprising one or more transport vehicles andasset management dashboards according to some embodiments of theinvention.

DEFINITIONS

To facilitate understanding of the invention, a number of terms aredefined below.

As used herein, the terms “processor” and “central processing unit” or“CPU” are used interchangeably and refer to a device that is able toread a program from a computer memory (e.g., ROM or other computermemory) and perform a set of steps according to the program.

As used herein, the terms “computer memory” and “computer memory device”refer to any storage media readable by a computer processor. Examples ofcomputer memory include, but are not limited to, random access memory(RAM), read only memory (ROM), computer chips, digital video discs(DVD), compact discs (CDs), hard disk drives (HDD), and magnetic tape.

As used herein, the term “computer readable medium” refers to any deviceor system for storing and providing non-transitory information (e.g.,data and instructions) to a computer processor. Examples of computerreadable media include, but are not limited to, DVDs, CDs, hard diskdrives, magnetic tape and servers for streaming media over networks.

As used herein, the term “in electronic communication” refers toelectrical devices (e.g., computers, processors, etc.) that areconfigured to communicate with one another through direct or indirectsignaling.

As used herein, the term “survey grade GPS” refers to global positioningsatellite (GPS) receivers that are able to map locations with a veryhigh degree of accuracy. For example, in some embodiments, survey gradeGPS receivers can measure an object's absolute position on the earth towithin 1 centimeter (0.4 inches) or lower.

As used herein, the term “fruit tree” refers to any perennial tree,bush, or vine that produces a fruit. The term “fruit” refers to a partof a flowering plant that derives from specific tissues of the flowerand is not limited to culinary fruits. Examples include but are notlimited to, abiu, acerola, almond, amla (indian gooseberry), apple,apricot, aprium, avocados, bael, bananas, ber (indian plum),blackberries, blood orange, blueberries, breadfruit, calamondin,cantaloupe melon, carambola (starfruit), cashew, the fruit, cherries,chestnut, chocolate, chokecherry, citron, cocoa, coconuts, coffee, cornplant, crabapple, cumaquat, currant, custard-apple, dates, dewberries,dragon fruit, durian, feijoa, fig, grapefruit, grapes, guava, hazelnut,honeydew, hops, jaboticaba, jackfruit, jujube, kaffir lime, key lime,kiwifruit, kumquat, lemons, limes, loganberries, longan, loquat, lychee,macadamia, mandarin, mangoes, mangosteen, medlar, morello cherry,mulberries, natal plum, nectarines, olives, oil palm, oranges, papayas,passion fruit, pawpaw, peaches, pears, pecan, persimmon, pineapples,plums, pluot, pomegranate, pomelo, pongamia, prune, pummel, pumpkin,raspberries, red banana, rock melon, sabine, sapodilla (chikoo), sapote,soursop, starfruit, stone fruit, strawberries, strawberry tree,sugar-apple (sharifa), surinam cherry, tamarillo, tamarind, tangelos,tangerines, tomatoes, ugli, uglifruit/uniqfruit, walnut, watermelon, agrape vine, a tomato vine, or an apple tree. In some embodiments, thefruit tree is a citrus tree (e.g., those described above).

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to systems and methods for monitoringagricultural products. In particular, the present invention relates toagricultural asset management, including: monitoring fruit production,tracking plant growth, monitoring plant vitality and productivity,measuring morphological attributes and performing field-basedphenotyping. By fusing data collected by a data acquisition componentand software component, embodiments of the present invention providesystems and methods for performing the foregoing.

The ability to accurately predict fruit yield and quality in advance ofharvest provides a significant advance for growers in the fruitindustry. The systems and methods of embodiments of the presentinvention provide an accurate inventory of crop holdings, including theimages and geo-location of each plant, as well as providing threedimensional models in the form of point clouds for data dissemination.In some embodiments the models can also be polygon models.

Embodiments of the present invention provide systems and methods forassessing agricultural crops (e.g., fruit trees, bushes, or vines). Thesystems and methods described herein find use in a variety ofapplications (e.g., providing information to companies engaged ingrowing fruit crops, those insuring fruit crops and those who invest inagricultural commodities).

In one aspect of the invention, the system is adaptable to monitor thephysical attributes of a wide variety of different species of fruitingplants, or different subspecies of fruiting plants. For example, becausesome plant varieties or cultivars are taller than other varieties, thesystem may be adapted and scaled to scan and collect data from trees andvines that have vast differences in height within the same row or columnor region of an orchard. In preferred embodiments, the LiDAR sensor ispositioned approximately 1.5-3 meters from the surface of the ground toprovide desirable range accuracy. In some embodiments of the invention,the LiDAR scanner is a push-broom scanner that collects data through anarray of detectors that are perpendicular to the direction of travel. Inthis way, the system may generate and geo-register point cloud data on amoving platform.

In some embodiments of the invention, the point cloud data may be usedto form a digital elevation model to accurately and precisely determinenot only a tree's morphology, but also, the lay of the land. Forexample, the point cloud data may be used to calculate a tree's canopyheight by determining the distance between vertices representing theground and vertices representing the tree's canopy crown.

The present invention is not limited to a particular fruit tree or vine.Plants that can be analyzed by using the systems and methods describedherein can include without limitation permanent crop plants, crop trees,forestry and fruit bearing plants. Examples of fruit bearing plantsinclude but are not limited to abiu, acerola, almond, amla (indiangooseberry), apple, apricot, aprium, avocados, bael, bananas, ber(indian plum), blackberries, blood orange, blueberries, breadfruit,calamondin, cantaloupe melon, carambola (starfruit), cashew, the fruit,cherries, chestnut, chocolate, chokecherry, citron, cocoa, coconuts,coffee, corn plant, crabapple, cumaquat, currant, custard-apple, dates,dewberries, dragon fruit, durian, feijoa, fig, grapefruit, grapes,guava, hazelnut, honeydew, hops, jaboticaba, jackfruit, jujube, kaffirlime, key lime, kiwifruit, kumquat, lemons, limes, loganberries, longan,loquat, lychee, macadamia, mandarin, mangoes, mangosteen, medlar,morello cherry, mulberries, natal plum, nectarines, oil palm, olives,oranges, papayas, passion fruit, pawpaw, peaches, pears, pecan,persimmon, pineapples, plums, pluot, pomegranate, pomelo, pongamia,prune, pummel, pumpkin, raspberries, red banana, rock melon, sabine,sapodilla (chikoo), sapote, soursop, starfruit, stone fruit,strawberries, strawberry tree, sugar-apple (sharifa), surinam cherry,tamarillo, tamarind, tangelos, tangerines, tomatoes, ugli,uglifruit/uniqfruit, walnut, watermelon, a grape vine, a tomato vine, oran apple tree. In some embodiments, the fruit tree is a citrus tree(e.g., those described above).

Permanent crop plants include without limitation trees, bushes or vinesthat, once planted, produce agricultural products that can be harvestedrepeatedly over multiple growing seasons. Permanent crop plants can beused to create products that include leaves and bark, such as forexample tea, cinnamon, cork and similar crops. They can also be used tocreate saps and other liquid crops, from plants such as sugar maple,birch, palm tree and other similar crops. Permanent crops can includefruiting trees, such as for example apple trees, pear trees, fig trees,olive trees, and banana trees; citrus trees such as for example, orangetrees, lemon trees, and grapefruit trees; stone fruit trees, such as forexample peach trees, avocado trees, and cherry trees; nut trees such asfor example pecan trees, walnut trees, and almond trees; and berrybushes, such as for example, blueberry bushes, raspberry bushes, andother similar crops. Permanent crops can also include vines, which caninclude, but are not limited to, table grapes, wine grapes, kiwi, andother similar crops.

Crop trees include trees that are harvested in total, one time, but takemultiple seasons of growth prior to harvest. These can produce suchdiverse agricultural products as pulp for paper, lumber, and decorativeuses such as Christmas trees and other similar uses.

In some embodiments, the present invention provides systems and methodsfor a) determining the diameter and/or circumference of a tree trunk orvine stem, determining the overall height of each tree or vine,determining the overall volume of each tree or vine, determining theleaf density and leaf color of each tree or vine; b) determining the GPSlocation of each plant and attaching a unique identifier (e.g., RFID tagor barcode identifier) to each plant or vine; c) determining thepredicted yield from identified morphology blossoms and/or fruit; and d)providing yield and harvest date predictions or other information to endusers using a user interface. In some embodiments, the technology isused to size fruit while it is still on the tree (e.g., for applicationswhere selective harvest is done based on size).

Embodiments of the present invention provide a variety of hardware andsoftware systems to perform the described methods. In some embodiments,systems comprise one or more (e.g., all) the following components:survey grade GPS, 3D laser scanners, static and motion imaging (e.g.,RGB, multi-spectral, hyper-spectral, NWIR and SWIR), high speed HDvideo, transport vehicles, computer software, computer processors, anduser interfaces.

The global positioning system (GPS) is a space-based satellitenavigation system that provides location and time information in allweather, anywhere on or near the Earth, where there is an unobstructedline of sight to four or more GPS satellites. GPS systems generally fallin one of four categories of accuracy, sub-centimeter (0.39370 inch),sub decimeter (3.937 inches), sub meter (39.37 inches), and subdecameter (32.8 feet). In some embodiments, survey grade GPS (e.g., subcentimeter) is used to locate fruit, and trees or vines to high levelsof detail. Survey grade GPS receivers determine location to very highaccuracy (e.g., 1 inch or less, 1 centimeter or less, etc.). GPS is atype of global navigation satellite system (GNSS). The GNSS in Russia isGloNASS, and the GNSS in China is known as BEIDOU.

In some embodiments, commercially available equipment is utilized. Forexample, survey grade GPS receivers are available from a variety ofsuppliers (e.g., Igage Mapping Corporation, Salt Lake City, Utah;Hemisphere Inc., Calgary, Alberta, Canada; Trimble Navigation Limited,Sunnyvale, Calif.).

In some embodiments, the systems and methods may include an inertialnavigation system (INS) to calculate sensor motion and orientation. AnINS may include one or more inertial measurement units (IMUs), which maycomprise accelerometers, gyroscopes, and magnetometers for providingrelative measurements of motion and orientation, such as for example,roll, pitch, or yaw. The motion and orientation measurements may be usedto improve the location measured by the data acquisition component. Inturn, the improved position measurements may be used to improve themapping of the point cloud data into geographic space. The INSadditionally provides tracking when satellite communications becomeunavailable, such as for example, when the equipment is under a treecanopy or similar obstruction.

In some embodiments of the invention, the INS may be coupled to orinclude a GNSS receiver. In this way, the INS may determine an initialposition, orientation, and velocity. As the transport component travelsthrough an orchard, its acceleration, orientation, and velocity may bemeasured by the INS and used to extrapolate its position based on theinitial position, orientation and velocity measurements. For example, ifthe GPS loses its signal with a satellite, the position of the transportcomponent may be estimated based on the acceleration, orientation, andvelocity measurements recorded by the INS.

In some embodiments of the invention, the position data measured by theINS and GPS may be processed for errors. For example, after thetransport component has gathered data, the data recorded by the GPS orINS may be analyzed to determine if there was an error in the GPS data(e.g., because of a canopy obstructing the signal to a GPS satellite).Errors may be identified by comparing the GPS data to the INS data(which may include data obtained from the IMU such as roll, pitch, oryaw data, or any combination thereof). If the GPS data suggests that thetrajectory of the transport vehicle shifted in one direction, yet theIMU data suggests the trajectory of the transport component did notchange based on its measured yaw, it may be determined that the positionof the transport vehicle is more accurately reflected by the readings ofthe IMU. The position of the transport vehicle may then be determined bygenerating a smooth best estimate trajectory (“SBET”), which isdescribed in more detail below.

In some embodiments of the invention, the INS and GPS may be integratedtogether, and may further include a second IMU integrated into thecircuitry of the GPS. The second IMU may then be used to correct the GPScalculations in the same manner as described above, and in real-time.The second IMU may measure physical movements of the device in at least3 axes, including pitch, yaw, and roll. The yaw axis measurement may beused to compensate for GPS signal outages or position errors. Whentraveling down a substantially straight trajectory, such as for examplewhen traveling through a row, the inertial navigation system may use theyaw axis measurement data to project the path of the collection vehiclealong the same substantially straight trajectory it was previouslytraveling on. In this way, the IMU data is used to smooth the positionoutliers by assuming the transport vehicle continued to travel along thesame substantially straight trajectory it was previously traveling on.In other embodiments of the invention, different trajectories may bemodeled using information known about the orchard or the route or paththe transport vehicle planned to take. For example, knowledge of thelayout of the orchard may include the geometry of the rows formed in theorchard. The layout of the circular or spiraled rows, may then be usedto model circular or spiraled trajectories of the transport vehicle.Similarly, knowledge of what path the transport vehicle is planned totake in an orchard may be used to extrapolate the transport vehicle'strajectory path.

In other embodiments of the invention, the INS further comprises a basereceiver station positioned in a fixed location proximal to the orchard.While the INS travels through the orchard, the INS may receive messagesfrom a GPS satellite which it may forward to the base station. In thisway, the INS functions similarly to a rover station in a real-timekinematic positioning system. While the INS transmits messages to thebase station, the base station in parallel receives the same messagesfrom the GPS satellite. Thus, the base station receives messages fromthe GPS satellite in sync with the INS. The base station may thencompare the GPS message received directly from the GPS satellite to theGPS message received from the INS. Based on this comparison, the basestation may determine whether the transport vehicle has deviated from atrajectory path based on its GPS data, and thus if there have been anyanomalies or errors with the INS. The messages the base station receivesfrom the GPS satellite describe the orbital information (ephemeris) ofthe satellites in view and the pseudo-range data (observables or epochs)describing the position of the other satellites in the GNSS with respectto the position of the base station. The base station will reliably knowthe geometry of the satellites because it will have a constant viewingpoint to them. As such, it is enabled to provide reliable positioncorrections to the rover when the rover has an outage or cycle slip.

In some embodiments of the invention, the orbits or position of thesatellites may be used to improve the accuracy of the data.Specifically, the precise ephemeris files which describe the exact orbitand position information of the satellites may be downloaded. The exactorbit and position information may then give precise positionmeasurements, improving positional accuracies to as little as a fewcentimeters. Having high accuracy at the resolution of centimetersenables the system to fuse sensor data and positional data with a highdegree of precision and accuracy. Further, errors that occur duringreal-time collection may be corrected, eliminating bias from poorsatellite coverage.

In some embodiments of the invention, when the base station determinesthat there has been a deviation, the base station may then compare theGPS data of the transport vehicle to the GPS data received directly fromthe GPS satellite. If the GPS data received directly from the GPSsatellite suggests a different trajectory path, it may be determinedthat the GPS data of the inertial navigation system is inaccurate. ASBET may then be generated to correct for the deviation, as described inmore detail below.

In some embodiments of the invention, the software component may use aweighting system for the positional data sources when generating theSBET. Specifically, if one positional data source is consistentlyunreliable, the software component may give that positional data sourceless weight when creating a SBET. For example, one GPS satellite may becausing a bias to the positional data. Knowing that the particular GPSsatellite is causing a bias, the software component may correct for thatbias by according that satellite less weight when calculating a geodeticposition.

A SBET may be created by performing a series of filtering algorithms onthe geodetic positional data calculated by the INS and GPS. For example,in some embodiments of the invention, the software component may apply aKalman filter to the geodetic positional data measured by the INS. Itmay then continue to filter the geodetic positional data by applying aforward, reverse, combine, precise forward, and then smoothingcorrection filter to the geodetic positional data, resulting in a SBET.Whereas the original/raw geodetic positional data may be accurate toabout 10 meters during a collection, in some aspects of the invention,the improved error identification and correction processes describedabove allow the INS to achieve a resolution of geodetic position data aslow as 1-3 centimeters in both the horizontal and vertical axes, and aslow as millimeters in the horizontal axis.

The processing of the position data described above may be performed bythe software component or data acquisition component, in real-time orafter the data has been collected. However, according to someembodiments of the invention shown in FIG. 9, the system may include apost-processing server 901 that performs the processing of the positiondata. The post-processing server 901 may receive position data from oneor more software components or data acquisition components 903 coupledto or mounted on one or more transport vehicles 904. The post-processingserver 901 may then provide the post-processed data to one or more assetmanagement dashboards 902. The post-processing server 901 may alsoprocess any other data collected, generated, or processed by the dataacquisition component and computer processor coupled to the transportvehicle.

Each position measured by the GPS and INS may be recorded and associatedwith a time stamp. As explained in more detail below, the time stamp maythen be used to fuse data measured by other sensors of the system. Insome embodiments, 3D laser scanners are utilized to map fruit or tree orvine properties. The term “3D scanner” is a colloquial term used todescribe a device used for light detection and ranging (LiDAR) which isan optical remote sensing technology that can measure the distance to,or other properties of a target by illuminating the target with light.LiDAR can be found in two common forms direct energy detection, alsoknown as incoherent, and coherent detection. Coherent systems aretypically preferred for measurement systems. Both systems are currentlyavailable in two pulse formats: micropulse and high-energy systems. Themicropulse systems are eyesafe and require less energy to operate butthis comes at the expense of higher computational post-processingrequirements. Some LiDAR systems currently in use are capable ofcollecting nearly one million points per second. The data collected isrepresented as a point cloud as demonstrated in FIG. 5. In a pointcloud, the individual points model the surface of every object in thesensor's range. For example, the point cloud may be parsed into modelsof leaves, branches, fruit, or entire trees. Three dimensional models,such as polygon mesh models may be derived from point clouds. 3D laserscanners are commercially available (e.g., from Tiger Supplies Inc.,Irvington N.J.; Laser Design and GKS Services, Minneapolis, Minn.; RieglUSA, Orlando, Fla. and Faro USA, Lake Mary, Fla.). In some embodiments,waveform LiDAR (e.g., available from Riegl, Orlando, Fla.) is utilized,and in yet other embodiments discrete LiDAR is utilized.

According to some embodiments of the invention that utilize discreteLiDAR systems, when a sensor emits a laser pulse, the laser pulse willtypically reflect off an object in the environment, and return to alight-detecting sensor. Several measurements may be taken on a singlepulse returned to the light-detecting sensor, such as for example, therange to the object, the sensor time, the angle relative to the sensor,and the intensity of the returned pulse. The sensor time corresponds tothe precise time the measurement was taken and is used to georegistermeasurements taken by the sensor. Specifically, the sensor time is thetime a recorded echo is received at a receiver. The position of thesensor is also correlated to the sensor time, allowing the measurementsto be related to the sensor position. In some embodiments, the sensortime may be determined based in part on attributes of the sensor. Forexample, a sensor may be configured to emit pulses at a predeterminedfrequency, such as for example, a certain number of pulses per second(“PPS”). This may be used to assist with the determination of theprecise time that a measurement was taken.

In some embodiments, a second pulse is primed and emitted at a specificinterval after the first pulse has been emitted. Discrete LiDAR systemsmay also record multiple returns from a single emitted pulse, indicatingthat the pulse reflected off multiple surfaces.

According to some embodiments of the invention that utilize waveformLiDAR systems, the returned laser pulse is separated into threedimensions: reflection; intensity; and pulse width. The extractedwaveform is fitted to a smoothed pulse waveform using a Gaussiantechnique, which provides a continuous measured signal. Whereas discreteLiDAR provides a sample of range data points, waveform LiDAR may provideinformation at each point along the surface of an entire object. Becausewaveform LiDAR produces a continuous signal, it is generally moreaccurate and provides more detailed measurements than discrete LiDAR.Specifically, the reflectance, intensity and pulse width of waveformLiDAR yields higher point density measurements, and captures theintricacies and details of foliage grown on trees.

Further, waveform LiDAR is capable of measuring calibrated reflectanceof a returned laser pulse which enables the system to isolate physicalattributes of a plant. The signal measured at the light-detecting sensoris converted to a digital signal, to produce an amplitude reading andobtain a measurement of the echo width defining the verticaldistribution of a target's surface as well the surface roughness of thetarget in question. The amplitude reading from the recorded waveform maythen be compared to the emitted waveform and normalized accordingly. Thenormalized amplitude reading may then be calibrated to yieldreflectance. The reflectance value associated with a particularamplitude reading is determined by utilizing a lookup table thatcorrelates range-dependent amplitude readings with reflectance. In someembodiments, the waveform LiDAR sensor performs on-board full waveformcalculations in real-time. In a preferred embodiment of the invention,the 3D laser scanner is configured to operate at a frequency of 550,000points per second, which yields an optimum density and coverage.

In some embodiments, high-speed, high-density (HD) video is used tocapture images of fruits and tree or vine features. The quality of videocaptured is important for accurate analysis. In some embodiments, videothat is uncompressed 1080p at a speed of 60 frames a second or faster isutilized. In some embodiments, a fisheye lenses of 160° or greater isutilized. In yet further embodiments, cameras capable of capturing videoof even higher definition or resolution can be used, such as forexample, Ultra High Definition or 4 k, which are capable of displayingat least 8 million active pixels. These cameras may also be used inconjunction with one another to provide depth information (e.g., stereovision) or various panoramas that cannot easily be achieved with asingle camera.

Infrared thermography (IRT), thermal imaging, and thermal video areexamples of infrared imaging science. Thermal imaging cameras detectradiation in the infrared range of the electromagnetic spectrum (roughly9,000-14,000 nanometers or 9-14 μm) and produce images of thatradiation, called thermograms. Since infrared radiation is emitted byall objects above absolute zero according to the black body radiationlaw, thermography makes it possible to see one's environment with orwithout visible illumination. The amount of radiation emitted by anobject increases with temperature; therefore, thermography allows one tosee variations in temperature. This is particularly useful when dealingwith plant species that have very dense leaf coverage since thetemperature differential between the leaf and the fruit is significant.The measurable temperature difference between the leaves and the ambientair can also provide important information about the vigor of the plant.High speed HD video hardware is commercially available (e.g., from NACImage Technology, Simi Valley, Calif.; Olympus, Tokyo, Japan;Panasonic). Thermal imaging equipment is commercially available (e.g.,from FLIR Systems, Boston, Mass. and L-3 Communications, New York,N.Y.).

In some embodiments, the capturing of this data requires specializedequipment mounted to the transport vehicle. In some embodiments, thetransport vehicle can be for example, a 4 wheel drive, flying vehicle,off road vehicle, or any other suitable ground-based vehicle. Thetransport vehicle may be, for example, a John Deere Gator™ UtilityVehicle. In some embodiments, the transport vehicle may be an airbornevehicle. The transport vehicle can be a manned or unmanned vehicle thatdrives on the ground or flies low to the ground to transport the mappinghardware and other equipment throughout the area to be surveyed.Unmanned transport vehicles can be self-driving or remotely controlled.In some embodiments, a single data collection unit is capable ofscanning two acres or more (e.g., 5, 10, 12, 15 or more) per hour interrestrial applications.

In some embodiments, the data acquisition component may include a userinterface as shown in FIGS. 7a, 7b, and 7c for controlling a dataacquisition component and transport component. According to theembodiments shown in FIGS. 7a, 7b, and 7c , the user interface mayinclude icons and selectable buttons for controlling the collection ofdata and displaying the status of the data acquisition component,transport vehicle, software, and hardware. For example, as FIG. 7ashows, the user interface may include a start button 701 for beginningthe collection of data, a stop button 702 for stopping the recording ofdata, an icon 703 for showing the speed of the transport vehicle, andone or more icons 704 that show the status of internet or wirelessconnectivity of the data acquisition component's wireless transceiver,and the signal strength of its GPS receivers. As shown in FIG. 7b , theuser interface may further include an upload button 705 for transferringthe collected data to a remote server. The icons shown in FIGS. 7a and7b may be depicted in different forms at different locations of the userinterface. As shown in FIG. 7c , the user interface may be provided on amobile device, such as a tablet, PDA, or smart phone. In someembodiments, the user interface may be on a remote system, or a centraldashboard computer.

In some embodiments, the present invention provides data analysissoftware and computer hardware configured to analyze data from the GPS,INS, scanners and video cameras described herein. In some embodiments,analysis systems include user interfaces and display systems. Forexample, the 3D scanner creates a point cloud which is a set of verticesin a three-dimensional coordinate system. These vertices are usuallydefined by X, Y, and Z coordinates, and are typically intended to berepresentative of the external surface of an object. When combined withpositional data from the GPS and INS, the vertices may be mapped onto ageodetic coordinate system.

In some embodiments of the invention, the different datasets describedabove are synchronized with each other using timestamps that indicatewhen the data was captured. For example, each location recorded by theGPS may be associated with a timestamp of when the location wasrecorded. Similarly, each LiDAR measurement recorded by the 3D laserscanner may be associated with a timestamp. The GPS and LiDAR data maythen be synchronized by matching the timestamps of the two datasets. Insome embodiments, the sensors are integrated into a single sensor systemand are coupled to a single clock which makes synchronization of thedata possible without using timestamps.

In some embodiments of the invention, the INS and GPS systems are usedto translate measurements taken by other sensors into real-worldgeodetic coordinates. As described in more detail below, measurementstaken by a sensor (e.g., a 3D LiDAR scanner) may be initially recordedin a scanners own co-ordinate system (“SOCS”). The SOCS is a coordinatesystem that uses the sensor as the reference frame, and all measurementscollected by the sensor are in relation to that sensor's referenceframe. For example, when the 3D LiDAR scanner generates a point cloud,the coordinates of each vertex in the point cloud are based on thesensor's SOCS. In order to translate the measurements from the sensor'sSOCS to real-world geodetic coordinates, the software component may usethe INS positional data. Specifically, the software component may usethe IMU coordinate reference frame to translate the SOCS coordinates togeodetic coordinates. For example, the software component may apply arotational matrix and X-Y-Z translation to perform this translation intothe IMU's reference frame. The rotational matrix may be a 3×3 rotationalmatrix. The software component may then use the time-stamps of the INSto correlate the SOCS coordinates of the sensor with real-world geodeticpositions. In this way, the software component may translate the SOCS ofthe sensor measurements into the reference frame of the inertialnavigation system. For example, using the known real-world coordinatesof the LiDAR scanner as determined by the INS and GPS, the softwarecomponent may then determine the location of any vertex in the LiDARscanner's SOCS. The high levels of accuracy achieved by the INS andsurvey grade GPS systems enable the software component to translatepositional information from a sensor's SOCS into real-world geodeticcoordinates with a sub centimeter level resolution. Thus, the preciselocation of a vertex in a point cloud may be mapped into real-worldgeodetic coordinates.

In some aspects of the invention, the orientation measurements obtainedby the IMU may be integrated into the analysis and mapping of pointcloud data in three-dimensional space. The IMU may be used to determinethe orientation of the sensor's reference frame, accounting for anyangular rotation or positioning of the sensor with respect to the othercomponents of the system. For example, if the 3D LiDAR scanner ispositioned at an angle with respect to the ground, the 3D LiDARscanner's SOCS will similarly be at such an angle. To account for theangular rotation of the data recorded in the SOCS, the softwarecomponent may apply rotation matrices to the SOCS that rotates and mapsthe SOCS data into real-world geodetic coordinates. By combiningorientation data with GPS positional data, the systems and methods maydetect skewed data points from poor registration to a mapping frame. Forexample, orientation data may detect when a tree is leaning in aparticular direction. With the integration of orientation data, thevertices of the point cloud may be calibrated to account for angles thatmay distort the 3D laser scanner data, resulting in a more accuratemapping of the vertices in three-dimensional space. In turn, theextraction of morphology attributes and phenotypes are more precise thanan extraction based on LiDAR and GPS positional data alone.

In some embodiments, point clouds of trees and vines are collected todetermine the height of the plant, the trunk diameter, and the leafdensity, each of which correlates with health and productivity and eachof which can be derived as part of an automated process calledclassification. The classified data is then loaded, another automatedprocess, into the database for access by the end user.

In some embodiments, software analyzes image data on a per tree or pervine basis. In some embodiments, data collected for a particular tree ororchard is inserted into a relational database providing connectivity toother industry standard software products. In some embodiments, softwareperforms one or more (e.g., all) of the following functions: a) assemblyof point cloud; b) correlation of data to map coordinates; c) refiningof video data; and d) geo-registration of all data that is collected,including for example, data from the survey grade GPS, the thermalimaging sensors, the radio, sound and magnetic wave sensors, themultispectral and hyperspectral sensors, the high-speed, high-density(HD) video camera, and point cloud data from the 3D laser scanner. Insome embodiments, the following information is provided to an end user:tree trunk diameter, height of tree, volume of tree, leaf density oftree, color of leaves on tree, GPS location of tree, bar code data fortree, number of blossoms on tree, number of fruit on tree, and anannotated or un-annotated photograph of the tree (see e.g., FIG. 1).

In one aspect of the invention, the software component processes thedata to automatically extract the morphological attributes of a tree.Models based on point cloud data may be used to extract morphologicalattributes, such as plant height or trunk diameter. As another example,the models may be used to perform segmentation and extraction ofstructural geometry. For example, the software component may search forboundaries of the three-dimensional point-cloud or other sensor data. Insome embodiments, the boundaries are identified by analyzing a changebetween two data point values. Algorithms that determine similarity andrate of change from data point to data point using features such ascolor, pattern, or distance may be used to classify points as belongingto specific objects. A change of more than a threshold amount mayrepresent an edge condition, which signifies the end of an object, orthe beginning of another. These edges mark boundaries that can be usedto classify each of the objects in an image or point cloud. Afterfinding a boundary, the software component may identify a canopy, andthen a trunk. The software component may identify a canopy by lookingfor a boundary or edge condition within a specific height range. Toautomatically extract a trunk, the post-processing server may attempt toidentify a series of arcs in the three-dimensional point-cloud. Thesoftware component may then attempt to fit the three-dimensionalpoint-cloud into a cylindrical shape. The portion of the point cloudhaving a best cylindrical fit may be identified as the plant trunk. Thesoftware component may then extrapolate plant trunk height.

In some embodiments of the invention, the morphological analysissegments and classifies features further, identifying branchingstructure, trunk segmentation, biomass estimation, tree stand structure,forest canopy height, extent, fruit clustering, and similar dimensionaland structural plant features. As described in more detail below, theseattributes may be used to determine the health and yield potential ofindividual trees and complete horticultural systems. These attributesmay also be used to determine genome, vitality, risk of damage fromwind, access to sunlight, and whether a plant has sufficient resourcesfor photosynthesis.

In some embodiments, the sensor data may be analyzed to classify andidentify various objects on a plant, such as fruits or blossoms. Forexample, variations in color or distances from sensor data may indicatethe presence of a certain material or object (e.g., an orange) in animage or point cloud of a citrus tree. In some instances, the variationsin color or distance may be caused by an overcast shadow or change insensor position. For example, an image of a citrus tree may include ashadow from fruit or branches from nearby citrus trees. When analyzingthe image, the software component may incorrectly detect the shadowedges and identify the shadow as a fruit. To account for such falsepositives, the software component may use the point cloud data toconfirm the presence of fruit. Specifically, if there is fruit, then thepoint cloud will be in the shape of the fruit and indicate the presenceof volume. However, if there is no fruit, the point cloud will berelatively flat, suggesting that the detected edge is merely a shadow.

In some embodiments, the variations are identified by analyzing a changebetween two pixel values. Algorithms that determine similarity and rateof change from pixel to pixel using features such as color, pattern, ordistance may be used to classify points as belonging to specificobjects. For example, a DD scan with noise may be used. As anotherexample, a Canny edge detection algorithm may be used. In yet furtherembodiments, the edge detection algorithms may utilize a Roberts edgeenhancement operator or Sobel edge enhancement operator. A change ofmore than a threshold amount may represent an edge condition, whichsignifies the end of an object, or the beginning of another. These edgesmark boundaries that can be used to classify each of the objects in animage or point cloud.

In another aspect of the invention, the software component processes thedata to extract the phenotypes of the tree. A plant's phenotype is thephysical expression of a plant's genetic heritage relative to itsdevelopment in its environment. A plant's phenotype may be, for example,the physical expression of the plant's genotype in combination withother information germane to its health and productivity. Theinformation germane to the plant's health and productivity may be, forexample, environmental stressors. A plant's phenotype is unique for eachtree. Thus, a citrus tree inflicted with a disease or suffering adrought will have a different phenotype than a healthy citrus treehaving the same exact genotype.

The physical expression of a plant's phenotype may be detected in partby its chemical composition. This, in turn, may be captured and measuredwith multi-spectral or hyperspectral sensors. For example, a citrus treeinflicted with a disease or suffering a drought may have a spectralsignature that is unique and distinct from a healthy citrus tree havingthe same exact genotype. The information germane to health andproductivity may be recorded for each tree. For example, environmentstressors such as temperature, weather, and similar microclimateconditions may be recorded for each tree. Such environment stressors maybe measured by coupling atmospheric sensors for measuring atmosphericconditions to the data acquisition component. In this way, an agronomistmay differentiate between plants that are unhealthy or unproductive dueto systemic causes or causes that are genetic/unique to the plant. Thisin turn allows growers to identify problems that may impact an entireorchard.

The software component may create a library of plant phenotypes thatspan across a wide array of plant cultivars and varieties which haveexperienced varying levels of health and productivity. The library maythen associate this information with the attributes that define thephenotype's physical expression, such as for example, a set ofhyperspectral or multispectral signatures, and a set of morphologicalattributes. Thus, when a grower wishes to retrieve information about aplant, the grower may for example access the plant's genotype, a set ofhyperspectral or multispectral signatures, and health and productivityconditions. The health and productivity conditions may include a historyof the plant's environmental conditions, such as for example, a historyof the plant's temperature, weather, and microclimate conditions.

In one aspect of the invention, a grower may use the library to analyzeor compare phenotypes of different plants or groups of plants atdifferent levels of granularity. For example, an agronomist may analyzethe phenotype of a single tree and compare it to the aggregatedphenotypes of the entire orchard. In this way, a grower may betteridentify and understand stressors that are impacting an orchard as awhole, and distinguish them from stressors that are impacting a singletree.

In some embodiments, the software component may rely on machine learningalgorithms (e.g., deep convolutional neural networks) to perform some orall of the analyses described above. In these or similar embodiments,the software is capable of learning from past experience. This enablesthe software to become more accurate over time by relying on a growingamount of data it has observed. These embodiments remove the need forparameters in algorithms that must be defined for accurate analysis andenable all or any combination of the data to be used together.

In some embodiments, the present invention provides methods of analyzingfruit tree quality and growth, including but not limited to, countingblossoms or fruit on the tree or vine (green or ripe), geo-locatingplant or trees, determining age and health of tree or vine based ontrunk diameter, leaf density, leaf color and overall volume. Thesemethods find use in a variety of research and commercial applications inthe agricultural, finance, banking, commodities, property appraisal, andinsurance industry.

In some embodiments, crop data is utilized to allow a user to predictcrop harvest by counting hanging fruit and measuring trunk diameter,leaf density, leaf color, environmental conditions, weather, previousharvests, age, variety, and overall volume. This is accomplished byutilizing a process that collects data in three formats: point cloud via3D laser scanning; geo-location data via survey grade GPS; andphotographic data via High speed HD video and/or thermal imaging.

In some embodiments, the systems and methods described herein find usein identifying subspecies of a particular species of tree or vine.

In one aspect of the invention, the point cloud data is fused with theother datasets, such as spectral data, to determine information aboutthe tree such as its health, ripening period, vitality, or age. Forexample, the software component may use the spectral data to calculatevarious vegetation indices across different spectral bands that provideinsight into the plant's health. Exemplary vegetation indices are setforth in Table 1 below.

TABLE 1 Vegetation Index Legend Range Normalized Difference Poor = <0.2Vegetation Index (“NDVI”) Moderate = >0.2 to 0.2.9 Good = 0.3.0 to 0.38Very Good = 0.39 to 1 Leaf Area Index (“LAI”) Poor = <0.5 Moderate= >0.5 to 0.9 Good = 1-1.67 Very Good = >1.67 Green Ratio VegetationIndex Poor = <2 (“GRVI”) Moderate = 2-3 Good = 3-5 Very Good = 5-8 GreenAtmospherically Resistant Poor = <0.1 Index (“GARI”) Moderate = 0.1-0.2Good = 0.2-.39 Very Good = 0.4-1 Soil Adjusted Vegetation Index Poor =<0.1 (“OSAVI”) Moderate = 0.1-0.15 Good = 0.15-0.25 Very Good = 0.25>The NDVI may be used as a measure of general vegetation health. As shownin the associated Legend Range, a higher NDVI indicates a healthierplant. The LAI may be used as a predictor of crop growth and yieldestimation. As shown in the associated Legend Range, a higher LAIindicates a higher crop growth and yield potential. The GRVI may be usedas an indicator of a plant's photosynthesis. As shown in the associatedLegend Range, a higher GRVI indicates a higher rate of synthesis. TheGARI may be used to indicate a plant's chlorophyll concentration. Asshown in the associated Legend Range, a higher GARI indicates a higherconcentration of chlorophyll. The OSAVI is similar to NDVI, butsuppresses the effects of soil pixels. The OSAVI is used in areas wherevegetative cover is low (i.e., <40%) and the soil surface is exposed,causing the reflectance of light in the red and near-infrared spectra toskew vegetation index values. As shown in the associated Legend Range, ahigher OSAVI indicates a better vegetation cover.

As shown in an exemplary embodiment as FIG. 6a , a tree's health may bedisplayed on a user interface and expressed numerically on a color-codedscale, with 1 indicating that the tree's health is poor and 5 indicatingthat the tree's health is above average. After determining each tree'shealth score, a map may be rendered showing each tree in a grove havinga color that correlates to the tree's health.

The fusion of data additionally facilitates the sizing and counting offruits or blossoms on a tree, thereby improving the prediction of atree's yield. For example, the fusion of red, green, and blue (RGB)spectral band data with point cloud data enables users to preciselyidentify clusters of point cloud vertices that correspond to fruits orblossoms. The determination of fruit size, quality, or counting may beaided by using a Shanon index, Gini coefficient, or machine learningalgorithms. The photogrammetric data, point cloud data, and spectral andthermal imagery are used to analyze factors such as plant color,geometry, and texture. Statistical and analytic functions may be used todifferentiate blossoms and fruits from leaves and branches. Machinelearning techniques may also be used to identify blossoms and fruits.For example, the vertices may be clustered using a density-based spatialclustering of applications with noise (“DBSCAN”) algorithm. The DBSCANalgorithm relies on a density-based notion of clusters which is designedto discover clusters of arbitrary shape. Clusters are defined as amaximal set of density-connected points. The parameters of the DBSCANalgorithm may be configured to specify a desired size of a neighborhoodand a desired density within the neighborhood. In other embodiments ofthe invention, the clusters may be identified using an octree K-meansisodata clustering algorithm to identify clusters.

In some embodiments of the invention, clusters of data points may beidentified by generating a training set based on one or more fruit orblossom attributes, and classifying a cluster based on the training set.For example, a training set may be generated using a grape vine havingseveral grape clusters. Various attributes of the grape clusters may bemeasured to complete the training set, such as the grape clusters' rangeof reflectance values, the range of pulse shape deviation, otherattributes related to full waveform data, and their spectral signatures.The software component may then use the clustering techniques describedabove to classify new grape clusters, but it may also utilize othermachine learning algorithms like neural networks.

The software component may use the cluster information to determine thesize and/or volume of fruit and blossoms on a plant. As shown on FIG. 8aor 8 b, the volume may be determined using a vertical cylinder method orby generating a convex hull/mesh based on point cloud data. In someembodiments, clusters can be classified by determining a thresholdnumber of point cloud vertices within a threshold area. For example, ifthere are more than 300 point cloud verticies within a 10 cm distance ofeach other, the area or volume encompassing these vertices may beidentified as a cluster. An octree K-means/Isodata clustering or similartechniques can be used to locate these clusters. In some embodiments ofthe invention, the volume may be further based on ground truth datacollected by a harvester. For example, if a harvester has measured theamount of volume or weight of fruit or crop generated by a specificplant, the software component may modify the volume estimation of acluster given the known volume production of the plant.

In some embodiments, the systems and methods described herein find usein detecting disease (e.g., through the addition of multispectral and/orhyperspectral sensors). Hyperspectral imaging works by the developmentof a digital fingerprint. Unlike a conventional digital camera, whichuses the three RGB bands, multispectral and hyperspectral imaging canuse data from a greater range of the electromagnetic spectrum.Multispectral imaging may have on the order of one to ten bands wherehyperspectral imaging may use tens to hundreds of bands. For example,there are approximately 250 sub-species of pecans in the US and thepecan tree has a particular signature in the electromagnetic spectrumand each of those sub-species have related but unique signature.

As described above, the software component may create a library ofplants that have experienced a diverse array of health or productivityconditions. The library may associate these plants with theircorresponding hyperspectral or multispectral signatures. The softwarecomponent may then use this library as a classified data set foridentifying correlations between various factors. For example, asdescribed in more detail below, the library may be used to detect thepresence of various conditions (e.g., diseases) in a plant given itshyperspectral or multispectral signature. That is, after the softwarecomponent determines the hyperspectral or multispectral signature of anew or unknown plant, it may compare it against the library ofhyperspectral or multispectral signatures. If the new plant's signaturematches the signatures of diseased plants for example, then the softwarecomponent may determine that the new plant has a specific type ofdisease.

The software component may compare the hyperspectral or multispectralsignatures of the new or unknown plant by deriving spectral informationdivergence of the signatures. The spectral information divergenceprovides a spectral similarity measure between two pixel vectors X andY. It quantifies the spectral discrepancy by making use of the relativeentropy to account for the spectral information provided by each pixel.The smaller the value of the spectral information divergence, thesmaller the differences between two spectral pixels. In a similar mannerthe morphological properties determined for a plant may also be used todetermine relative plant health or cultivar and sub-cultivar.

In some embodiments of the invention, the hyperspectral sensors mayinclude active sensors. Passive hyperspectral sensors typically measurespectral data based in part on reflected sunlight. Thus, hyperspectraldata is typically gathered during the day time and when light conditionsare at their peak. Active hyperspectral sensors, by contrast, typicallyprovide an independent source of energy or illumination, and a sensorfor detecting its reflected energy. In some embodiments, thehyperspectral sensor may include a sourced laser, operating between 900nm and 1650 nm with 4 nm bands. Because active sensors are equipped withan independent source of light, active hyperspectral imaging may beperformed at night or in areas that lack or are blocked from sunlight.Several advantages to performing hyperspectral imaging of trees andvines at night exist. For example, the quality or availability ofspectral data retrieved from passive hyperspectral sensors may beimpacted by shadows or weather conditions. Active hyperspectral sensors,by contrast, are not impacted by such variances, and allow growers tocollect hyperspectral data at night or under a wide range of weatherconditions. As another example, thermal imaging may be used todifferentiate fruits from dense canopy and leaf coverings at certaintimes in the evening better than during the day. Specifically, becausethe water in the fruit causes its temperature to differ from thetemperature in the leaves, the fruit are easier to discern from thefoliage of the tree during the cooler temperatures of the night insteadof the warmer temperatures of the day.

In some embodiments of the invention, the spectral data of the plantsmay be used to determine certain plant conditions such as its Nitrogenor chlorophyll content, the temperature of the leaves and hence thewater status, or its sub-species. For example, the temperaturedifference between the leaves and the ambient air provides informationabout the level of water stress the plant is under.

As explained above, the health or productivity of a plant may beaffected by certain stressors. Stressors may include internal factorsthat impact the functioning of the plant's systems, such as infection,infestation, injury, or poisoning. They can also include externalfactors that limit the plant's available resources, such as post-harvestgrowth, weed infestation, drought, and soil depletion. Some plants'reactions to stressors results in the diversion of specific chemicals toaffected areas, which change the spectral signature of the plants.Plants may also respond with a physical change, such as wilt or leafdrop, causing differences in spectral signatures from the orientation oftheir leaves, the shape and edge conditions of their leaves, and theircanopy density.

In the case of diseases such as “citrus greening”, “blight” or “citruscanker,” specific conditions are manifested in the leaf, trunk and/orfruit that change their spectral signature making them identifiablethrough machine vision techniques. Some diseases my also be observed bythe changes they produce in the morphological characteristics of aplant. Citrus greening, for example, may lead to a less dense canopycompared to healthy trees. This reduction in canopy density may then beobserved in the morphological features extracted from the data.Additional details are described, for example, in Lan et al., AppliedEngineering in Agriculture Vol. 25(4): 607-615 and Kumar, Arun, et al.“Citrus greening disease detection using airborne multispectral andhyperspectral imaging.” International Conference on PrecisionAgriculture. 2010. Each of these references are herein incorporated byreference in their entirety.

Blight is a wilt and decline disease of plants that when present incitrus trees (“Citrus Blight”) can cause them to become unproductive,exhibit twig dieback, off-season flowering, and the production ofsmaller fruits. Symptoms typically associated with citrus blight includea mild wilt and grayish cast to the plant's foliage, high zinc contentin trunk bark and wood, the presence of amorphous plugs in the xylem,the failure to absorb water injected into the trunk, and the presence ofblight-associated proteins in roots and leaves. Blight is typicallytransmitted by root grafts; however, it is not typically transmitted bylimb grafts or with budwood. Trees on all rootstocks are susceptible toCitrus Blight, but significant differences between stocks exist.

Citrus Blight is typically diagnosed on an individual tree in the fieldby testing water uptake into the trunk using a battery-powered drill anda plastic syringe without a needle. Trees affected by Citrus Blight takeup no water regardless of the amount of pressure applied. Forconfirmation of Citrus Blight using the serological test, small numbersof samples of mature leaves may be collected and sent to a diagnosticlab.

Diseases can cause a wide variety of symptoms that affect themorphological and/or spectral signature of a plant. For example, citruscanker may affect plant production by causing defoliation, shootdie-back, and fruit drop. Symptoms include lesions and blemishes in theleaves, stems, and fruits of a citrus tree. Lesions may appear raised onthe surface of the leaves, and in particular on the lower leaf surface.Pustules may also form on the surface, and eventually become corky andcrater-like, with raised margins, sunken centers and surrounded by ayellow halo.

Certain types of citrus fruits are more susceptible to some diseasesthan others. For example, grapefruit, Mexican lime, and some earlyoranges are highly susceptible to canker. Navel, Pineapple, Hamlinoranges, lemons, and limes are moderately susceptible to canker, whilemid-season oranges, Valencias, tangors, and tangerine hybrids are lesssusceptible. Tangerines are generally tolerant to Citrus Canker.

Citrus Canker outbreaks generally occur when new shoots are emerging orwhen fruits are in the early stages of development. Because thebacterium reside and reproduce in the lesions formed on the leaves,stems and fruit, the spread of Citrus Canker may be exacerbated byvarious weather conditions. For example, moisture or rain that collectson the lesions may cause bacteria to ooze out and spread to new growthor other trees. Weather conditions, such as frequent rainfall and warmweather, may also contribute to the prevalence of particular diseases.Wind-driven rain in particular is a significant dispersal agent of somebacterium. Wind speeds greater than 18 mph may cause the bacteria topenetrate through stomatal pores or wounds made by thorns, insects, andblowing sand. Heavy wind speeds may also cause the spread of thebacterium over greater distances. While long-distance spread of CitrusCanker may be caused by strong winds, the spread may occur more commonlywith the movement of diseased plant material by growers and theiremployees. Specifically, workers and grove equipment can spread thebacteria within and among plantings, especially when trees are wet.

If major rainfall occurs during the critical time period that new shootsare emerging or when fruits are in the early stages of development, thelikelihood of a Citrus Canker outbreak emerging grows significantly.Typically, leaf and stem infections occur within the first 6 weeks afterinitiation of growth, unless the plant has been infected by Asian leafminers. Similarly, fruit are particularly vulnerable to infection whenthey are between 0.5-1.5 inch in diameter for grapefruit and 0.25-1.25inch in diameter for oranges. After petal fall, fruit generally remainsusceptible to Citrus Canker during the first 60 to 90 days for orangesor tangerines and 120 days for grapefruit.

Citrus Greening (also known as “Huanglongbing”) is generally caused bythe bacterium Candidatus liberibacter asiaticus. Like Blight and CitrusCanker, Citrus Greening affects tree productivity and causes treedecline. Root systems of infected trees are often poorly developed, andnew root growth may be suppressed. Affected trees may show twig dieback,causing their productivity to decline, or stop altogether. The diseasemay impact the fruit produced by the tree, causing the fruit to be fewerin number, smaller, lopsided with a curved central core, discoloration.The disease may additionally cause fruit to drop prematurely from theafflicted tree. The affected fruit often contain aborted seeds and havea salty, bitter taste. When psyllids are abundant and conditions arefavorable, the disease can spread, destroying existing groves andpreventing the commercial production of oranges and other citruscultivars.

Early symptoms associated with Citrus Greening include vein yellowingand an asymmetrical chlorosis (referred to as “blotchy mottle”) onleaves, smaller-sized and upright leaves, and leaves that exhibit avariety of chlorotic patterns that often resemble mineral deficienciessuch as those of zinc, iron, and manganese. Normally symptoms are severeon sweet orange, mandarins, and mandarin hybrids; and moderate ongrapefruit, lemon, and sour orange. Lime, pummelo, and trifoliateoranges are generally more tolerant. Additionally, the bacterium may bediagnosed by a Polymerase Chain Reaction (PCR) from symptomatic tissues.

In one aspect of the invention, the spectral data may differentiatebetween trees with diseases such as Citrus Greening, trees in decline,trees that are deficient in nutrients, and trees that are healthy basedon the measured reflectance of the tree across certain spectral bands.For example, a tree with Citrus Greening may be characterized by aunique combination of reflectance values ranging from 10% to 40% acrossthe 1200 to 2000 nm wavelength band. A healthy tree, by contrast, has auniquely distinct set of reflectance values across this band. After thereflectance values for a particular plant across this wavelength bandare measured, the software component determines whether the reflectancevalues are associated with a healthy tree or a tree infected with CitrusGreening.

In some embodiments of the invention, the use of spectral imaging allowsgrowers to determine the difference between a leaf exhibiting adeficiency in a nutrient, such as for example, zinc, and a leaf that ishealthy or diseased.

In some embodiments, the software component may be configured to performa Brix analysis, allowing the growers to evaluate the sugar content offruit throughout its development. Brix may be measured in degrees (° Bx)which corresponds to a hydrometer scale that determines the amount ofsugar in a solution at a given temperature. The measure is relative tothe concentration of one gram of sucrose dissolved in 100 grams of water(1° Bx). The ° Bx may be associated with the ripeness or quality of afruit. For example, harvest-fresh pineapple has a Brix measurement ofapproximately 13° Brix. Natural degreening of pineapple may occur whenpineapples are transported. This degreening process is not visible fromthe outside, however, may be detected by changes to its Brixmeasurement. Typical measurement values for a harvest-fresh pineapple ondelivery are 14.2 to 14.7° Brix. If the measurement value is clearlybelow this range, the fruit is poor quality. High Brix values indicate asweeter taste and that the fruit or vegetable will keep for longer.

By fusing LiDAR with other data sets, such as multi- or hyperspectraldata, a grower is enabled to track disease vectors and determine how acondition is spreading across their property or throughout a specificregion. A vector may be an agent (e.g., person, animal, ormicroorganism) that carries and transmits a pathogen into anotherorganism. By mapping vectors and comparing them from time period to timeperiod, it is possible to determine how a condition is spreading throughan orchard, and postulate ways to minimize further transmission. Thetime periods for comparison may be for example, at the beginning or endof a growing season. In some embodiments, a grower may map the vectorsat more frequent intervals, such as, for example, a monthly, weekly, ordaily basis. The direction of travel of the vector may be identifiedusing statistical methods and/or spatial models to analyze and predictthe continued path of the vector. For example, maps of plants affectedby a particular disease, such as for example, Citrus Blight, may begenerated as a function of time. The maps may then be viewed together toshow how the condition has spread over time across particular regions ofa grower's orchard or plot.

In one aspect of the invention, a grower may be able to detect and trackthe presence of arthropod pests, such as aphids or spider mites. Pestsmay be detected and tracked by fusing LiDAR data with spectral data andthermal data where pests may be directly detected by their body orstructures created by the pest or indirectly by observing physiologicalchanges in a plant resulting from the pest. The degree of infestation,the rate of growth and potential direction of spread may be predictedbased on spatial associations of plant morphologies, sub-species,terrain, or environmental conditions.

In one aspect of the invention, a history of each type of data (e.g.,multi- or hyperspectral data) is associated with each plant, allowinggrowers to perform temporal analyses on each plant. Other attributes anddata that may be temporally recorded for each plant may include theplant's date of planting, its variety, its root stock, its graftingrecord, its nutritional treatment, its health treatments, its yield, itssurrounding soil conditions, its surrounding temperature, weather, andmicroclimate conditions, and topographical information. In someembodiments, a grower may define a custom data field that may betemporally monitored for each asset. The custom data field may includedata entered manually in the form of alphanumeric, photographic or otherdata types, as well as calculated fields such as indices or predictions.The data fields may also be automatically maintained. For example, ifthe data field corresponds to data collected by a sensor, the data fieldmay be automatically populated by data collected via a sensor. Some datafields may be populated from third party sources, such as for example,weather or soil readings. Other data fields may be manually entered onan ad hoc basis, and may be appropriate for monitoring activities suchas remedial pruning or weed removal. As described in more detail belowin reference to FIGS. 6a, 6b, and 6c , a history of grafting records forone or more plants may be viewed on a user interface that displays aninteractive map of the plants in the grower's orchard.

In another aspect of the invention, a grower may monitor which plantsits employees are handling. In this way, if a grower has identified aplant or area of plants as infected with a disease, the grower can limitor prevent an employee from coming into contact with another plant orarea, to prevent the spread of disease.

In yet another aspect of the invention, a grower may track storm anddominant wind profiles in a plant's history. A “dominant wind profile”is a history of wind speeds and direction in an area surrounding a plantor orchard. This information be used to help predict and track thespread of diseases. In some embodiments, the storm and weather profilesmay be used to predict and track the spread of diseases. For example,because rainfall and winds exacerbate the spread of Citrus Canker, pastrecords or patterns of rainfall or wind gusts may be used to determineor predict whether Citrus Canker in one plant may have been dispersed toanother plant or area.

In some embodiments of the invention, the data is used to extract healthinformation below the level of the individual plant. In suchembodiments, the health of individual branches or sections of the plantmay be determined. For diseases that spread from one part of theorganism to the others, this information may allow diseased portions ofa plant to be removed before they impact the health of the rest of theplant.

As described above in reference to plant phenotypes, a library may becreated that catalogues an assortment of plant cultivars and varietieshaving experienced varying levels of health and productivity, along withtheir associated physical attributes such as spectral signatures orplant morphology. In some embodiments of the invention, a grower mayidentify an unknown plant cultivar or variety using the informationstored in the library. For example, a grower may compare the spectralsignatures of the unknown plant to the spectral signatures stored in thelibrary. Specifically, the plant's measured spectral data may becompared to the spectral data of each plant at various wavelengths inthe library of cultivars and varieties to find a match. Any otherinformation known about the foreign plant may also be used to furtherassociate the unknown plant with a cultivar or variety. For example,environmental conditions, morphological attributes, or information aboutwhether the plant is diseased may be used to confirm an association witha cultivar or variety. In this way, variety and cultivar information foreach plant on a grower's property may be provided on a plant-by-plantbasis.

In one aspect of the invention, the crop yield for a grove may bepredicted based on the counting of fruits and blossoms as describedabove, as well as taxonometric information. Taxonometric informationhelps differentiate plants of certain subspecies from the overallspecies, and helps distinguish one plant variety from another.Taxonometric information includes any traits for categorizing themembers of the dataset. For example, if the plant or fruit's conditionis determined to be unhealthy or infected as described above, it willnot be included in the predicted crop yield. Similarly, if theenvironmental conditions of the plant are characterized by drought orextreme temperatures, the predicted yield may be lowered according tothe severity of the drought/temperatures. In some embodiments, predictedyield is calculated for each specific tree. As described in more detailbelow, the yield for each tree may then be aggregated at variousdifferent levels of granularity, such as by row, column, block, grove,geographic region, or market.

In some embodiments of the invention, the overall yield may be predictedbased on the attributes of multiple plants, such as the leaf area indexor canopy biomass. These are indicators of a plant's (or stand ofplant's) ability to collect sunlight, and, by extension, produce sugars.Attributes such as branching patterns may be used to indicate variety orinjury, which may affect yield.

In some embodiments, identification of subspecies or disease isperformed simultaneously with the other data that is being collected(LiDAR, photos, etc.) and geo-registered via GPS along with the otherdata.

In some embodiments, the data collected using the systems and methodsdescribed herein finds use in guiding sprayers through real-time mappingor premapping (e.g., commercial sprayers). Spraying represents a verylarge part of the budget of a grower and using the right kind of spray(herbicide, pesticide, and/or nutrient), in the proper amount, at theright time can have a dramatic impact on the farmers profitability. Insome embodiments, data is collected using the systems and methodsdescribed herein (e.g., trunk centroid, tree height, canopy density,canopy diameter, species of tree, longitude and latitude) are used tocontrol the sprayers. This data (e.g., in csv format) tells the sprayeras it travels through the grove from precalculated data, on a per treebases, when to spray, how long to spray, and what chemicals to spray.

In some embodiments, a prescription file is attached to a GIS shapefileusing the GPS, point cloud data and classification data described above.The prescription file further includes spraying instructions thatspecify the location and type of spray to be applied to each tree. Insome embodiments, the prescription information may include the spatialdata, point cloud data, and classified data described above, such astree height, canopy diameter, or leaf density, as well as otherinformation such as morphological attributes, spectral data,cultivar/variety, age and phenotype. For example, a prescription filemay contain a three-dimensional shape of a tree, the precise geographiccoordinates of the tree, an instruction as to what spray to apply to thetree, and which part of the tree to spray. In this way, the spray may beapplied to a particular tree or space within or surrounding a tree basedon the sprayer's precise geographic location. In one aspect of theinvention, a grower may control the application of a spray to aparticular tree based on the extracted morphological attributes,spectral data, cultivar/variety, age and phenotype of the tree. Forexample, the grower may configure the prescription file to apply a firsttype of spray to a tree that is identified as a 80 year-old Oconee pecantree, whereas the same prescription file may be configured to apply asecond and distinct type of spray to a 45 year-old Summer pecan tree inthe same row.

Some embodiments of the invention use the data to provide valuations ofplants. The morphological and health properties determined for theplants provide helpful information when assessing the value of a pieceof property and the plants on that property. This information can beuseful to individuals looking to buy or sell property containing plantsor for insurance companies looking to insure plants. Similarly someembodiments of the invention may be used to measure the status of plantsafter damage (e.g., after severe weather events). In such embodiments,the amount of damage can be quantified based on changes in the plantmorphology and health. In one example, the amount of a tree canopyremaining after a catastrophic weather event can be measured along withthe angle between the trunk and the ground. This information can be usedto assess about how much value was lost after damage occurs and could beuseful to entities like (but not limited to) insurance companies.

In some embodiments, the present invention provides computer implementedsystems and methods for performing fruit tree analysis and displayingthe results to a user. In some embodiments, computer implemented systemsand methods generate a report of the results of the analysis methodsthat provide information (e.g., fruit yield, tree quality, harvest datepredictions, sprayer coordinates) to a user. In some embodiments, thereport is provided over the Internet (e.g., on a smart phone, tablet orother wireless communication device) or on a computer monitor.

In some embodiments, the systems and methods of the present inventionare provided as an application service provider (ASP) (e.g., can beaccessed by users within a web-based platform via a web browser acrossthe Internet; is bundled into a network-type appliance and run within aninstitution or an intranet; or is provided as a software package andused as a stand-alone system on a single computer). For example, asshown in FIG. 9, some embodiments of the invention may include apost-processing server 901 that performs the processing of data receivedfrom one or more software components or data acquisition components 903mounted on one or more transport vehicles 904.

In some embodiments of the invention, the systems and methods include anasset management dashboard for accessing, viewing, analyzing andcontrolling the collected and analyzed data described above. Someembodiments of the asset management dashboard may be provided over theinternet, in the form of a web page, as shown in FIG. 6a . The assetmanagement dashboard may be used to render visualizations of the datacollected by the data acquisition component or stored in apost-processing sever. For example, the visualization may include a mapof the three-dimensional point clouds or a two-dimensional bird's-eyeview of the orchard 601, or both. As described above, the map may renderplants at a sub centimeter resolution, as well as any data collectedfrom other sensors, such as the 3D LiDAR scanners, spectral sensors,video sensors, or thermal sensors, and any calculations or analysesassociated with the plant, such as calculations or analyses related tothe plant's morphology or phenotype 602. For example, as shown in FIGS.6a and 6b , each plant rendered on the map may visually indicate thenumber of fruit or blossoms counted on that plant 602. In this way, themap may then be used to analyze the productivity of plants within theorchard or across a whole growing operation.

The asset management dashboard may further render a map of yieldestimates at a sub centimeter resolution. As described above, yield maybe estimated for a plant based on the three-dimensional point cloud dataor other data collected by the various sensors of the data acquisitioncomponent, or any combination thereof. These estimates may then bedisplayed on a map with the other calculations described above.

In one aspect of the invention, the asset management dashboard may beused to view the health status for each plant in its property. Forexample, as shown in FIG. 6a , a tree's health may be expressednumerically on a color-coded scale 603, with 1 indicating that thetree's health is poor and with 5 indicating that the tree's health isabove average. After determining each tree's health score, a map may berendered showing each tree in a grove having a color that correlates tothe tree's health. FIG. 6a illustrates a two-dimensional bird's eye viewof the visualizations, however, in other embodiments of the invention,the color-coded scales may be overlaid onto the three-dimensional pointcloud data for a three-dimensional visualization. For example, aninteractive map showing several trees colored as red indicates that thetrees in the grove are unhealthy. The scale thus provides a way forusers to quickly recognize trees that require attention without detailedanalysis.

In other embodiments of the invention, as shown in FIG. 6b , each plantmay be associated with one or more layers sensor of data and detailedcalculations and/or analyses 602. The additional layers of sensor datamay be for example, photographic, video, or spectral sensor data. Eachof the one or more layers of sensor data may comprise a plurality ofdata points that are geo-registered with the geodetic positions in thesame manner as described above. Each data point in an additional layerof sensor data may be associated with one or more vertices of thethree-dimensional point cloud. The asset management dashboard may thenoverlay the additional layers of sensor data on the three-dimensionalpoint cloud. As shown in FIG. 6b , a user may select a particular plant,triggering the asset management dashboard to display the data or subsetsof data that have been calculated and associated with the plant. Thedata may include morphological attributes, such as its trunk diameter orarea, as well as geodetic positional data such as the latitude andlongitude of the plant's centroid.

The asset management dashboard may also be configured to display asingle layer of sensor data and detailed calculations and/or analyses.For example, as shown in FIG. 6c , a single layer may be the NDVI thatis generated based on spectral data collected by the hyper-spectral ormulti-spectral sensors described above. The NDVI data may then beoverlaid on the same visualization of the plants shown in FIGS. 6a and6b , allowing agronomists to easily analyze correlations between yieldand spectral data for each plant. A legend 604 may be used to indicatethe NDVI associated with each tree.

As another example, red, green, and blue (RGB) spectral band dataobtained from a video sensor such as the video camera, may be overlaidwith three-dimensional point cloud data to precisely identify clustersof point cloud vertices that correspond to fruits or blossoms. In thisway, the fusion of data additionally facilitates the sizing and countingof fruits or blossoms on a tree, thereby improving the prediction of aplant's yield. The photogrammetric data, three-dimensional point clouddata, and spectral and thermal imagery may also be used to analyzefactors such as plant color, geometry, and texture. Statistical andanalytic functions may be used to differentiate blossoms and fruits fromleaves and branches as described above.

In one aspect of the invention, the dashboard may integrate the spatialinformation about a plot to allow users to generate precise counting andstatistical reports about the number of trees or vines and their health.The dashboard may further include one or more filters 605-612 forfiltering the plants visualized in the user interface or provided in thereport. For example, the dashboard may include a search box 605 thatallows a user to query the precise number of trees that are healthy in aparticular block, column, row, or geographic region. The dashboard mayfurther allow a user to use his or her mouse to select an area (e.g., arectangular region) on the map, and determine the number of healthytrees inside the region. The dashboard may include a health filter 606comprising the color-coded key 603 that correlates plant health score tocolor. Under each health score, the dashboard may show the number oftrees or vines with that particular score, and a selectable button forviewing which trees have that particular score. For example, when theuser selects the “View” button 607 shown under the Health Score labeledas “5” and colored as green, the map is re-rendered to only show thetrees that have a health score of 5, which FIG. 6a indicates is a totalof 824 trees. The dashboard may also allow a user to submit a query todetermine the health status for a particular tree by clicking on a treein the rendered map, or submitting the tree's block, column, and rownumber.

In some embodiments, the dashboard may integrate spatial informationabout a plot with the extracted morphological attributes, phenotypes, orclassification data described above. For example, the dashboard mayinclude a window that shows the number of trees according to their trunksize 608. As shown in exemplary FIG. 6a , a window labeled “Trunk Size”shows that there are 1524 trees with trunk sizes less than 2 incheswide. Under the total number of trees with this trunk size is aselectable “View” button that, when clicked, re-renders the map to onlyshow the trees that have a trunk size of less than 2 inches. Similarly,the dashboard may include a window that categorizes trees according totheir leaf density 609, and other morphological attributes, such asheight 611, and canopy diameter 612.

In one aspect of the invention, the dashboard may allow users to filtertrees according to different combinations of selected morphological,phenotype or health attributes. For example, the dashboard may allow theuser to query trees having both a trunk size of less than two inches anda health score lower than 2. In some embodiments, the user's selectedcriteria may be combined as logical Boolean queries. For example, thequery the dashboard may allow the user to query trees having a healthscore lower than 2; and either a trunk size less than two inches or aleaf density smaller than 5. For example, a grower could look at trunks<3″, canopy <6′, and health <3, to see trees that may have a healthproblem that is stunting their growth.

As shown in exemplary FIG. 6a , the dashboard may further include awindow 610 that allows users to record information about each plant'sage, treatment, or other notes that are specific to a plant. When a userselects a plant, the asset management dashboard may display a scrollablepop-up window or menu with a table of information about each plant'sage, treatment, or other notes that are specific to the plant.

As described above, some embodiments of the invention allow thepredicted yield to be calculated for each tree within a grower'sproperty. Using the asset management dashboard, a grower may aggregatethe predicted yield data to determine the yield of a crop at variouslevels of granularity. In one aspect of the invention, the dashboard mayallow a user to seamlessly and instantly alternate between high-levelsof granularity that provide information about an orchard as whole anddiscrete levels of granularity that provide information about anindividual tree. For example, a user may aggregate the predicted yieldfor each tree in a row, column, block or grove 613. As another example,the user may select a particular region of the grove, by for exampleselecting an area on the map with his or her mouse, and querying thepredicted yield for the aggregate of trees in the selected region. Inthe same interface, a user may then select information about anindividual tree. In this way, a user may draw comparisons between plantsat different levels of scale that were previously not possible. In otherembodiments of the invention, the user may submit a query for predictedyield in conjunction with other selected criteria, such as for example,the predicted yield for a particular subspecies or cultivar/variety offruit.

In some embodiments of the invention, the predicted yield may be used inconjunction with methods and systems for measuring the actual yieldproduced by a tree. For example, the predicted yield may be compared tothe actual yield of a windrow using Leddar as described in U.S.application Ser. No. 15/209,689, entitled “SYSTEMS AND METHODS FORDETERMINING CROP YIELDS WITH HIGH RESOLUTION GEO-REFERENCED SENSORS” toK. Thomas McPeek, herein incorporated by reference in its entirety filedon Jul. 13, 2016. As another example, the actual yield may be measuredusing load cell weight measurements. The actual yields measured at aparticular location in the windrow may be assigned to specific plants orregions within a grower's holdings. In this way, a grower may compareand verify the predicted yields for each plant to actual harvest data,and further, to alert growers to underperforming assets.

In one aspect of the invention, data may be collected and aggregatedacross properties owned by different growers. The aggregated data may bemaintained anonymously and then used to generate yield predictions for amarket as a whole. Market wide predictions may then be distributed togrowers, financial organizations, commodities traders, governmentaldepartments, agricultural and environmental agencies, or other partieswith an interest in food prices.

All publications, patents, patent applications and accession numbersmentioned in the above specification are herein incorporated byreference in their entirety. Although the invention has been describedin connection with specific embodiments, it should be understood thatthe invention as claimed should not be unduly limited to such specificembodiments. Indeed, various modifications and variations of thedescribed compositions and methods of the invention will be apparent tothose of ordinary skill in the art and are intended to be within thescope of the following claims.

What is claimed is:
 1. A plant analysis system, comprising: a) a dataacquisition component configured to: measure properties of a plantutilizing waveform light detection and ranging (LiDAR) to assemble pointcloud data, wherein said point cloud data comprises a plurality of threedimensional vertices each of which represents an external surfaceassociated with said plant; collect photographic data of said plant;collect spectral data for said plant; and geo-register the point clouddata and photographic data; wherein the plurality of said threedimensional vertices of said point cloud data is associated with globalpositioning satellite (GPS) data, said photographic data, and saidspectral data; b) a transport vehicle on which said data acquisitioncomponent is mounted and configured to assemble and geo-register saidpoint cloud data, and collect said photographic data and said spectraldata on said plant; and c) a processor configured to: determine aspectral signature of said plant based on said spectral data; determineplant color based on said photographic data and associate said pointcloud data with said plant color; and generate morphological data ofsaid plant based on said point cloud data, said morphological datacomprising one or more of plant stem diameter, plant height, plantvolume, and plant leaf density.
 2. The plant analysis system of claim 1,wherein: the processor generates said morphological data by segmentingsaid point cloud data to identify boundaries of said plant; or theprocessor classifies said morphological data to identify a plantfeature, said plant feature comprising a branching structure, trunk,biomass, canopy, fruit, blossom, fruit cluster, or blossom cluster. 3.The plant analysis system of claim 1, wherein the processor utilizessaid point cloud data and said plant color to discriminate a fruit,blossom, fruit cluster, or blossom cluster from a shadow based, at leastin part, on analyzing a variation of said plant color and a geometricshape defined by the three dimensional vertices of said point clouddata.
 4. The plant analysis system of claim 1, wherein said dataacquisition component is further configured to measure atmosphericconditions, and wherein said processor determines a phenotype of theplant based on said spectral signature, said morphological data, andsaid atmospheric conditions.
 5. The plant analysis system of claim 1,wherein said processor is further configured to determine a vegetationindex of one or more plants based on said spectral data.
 6. The plantanalysis system of claim 1, wherein said processor is further configuredto determine a number and a size of fruits or blossoms on the plant,wherein the number and size of fruits or blossoms on the plant is basedone or more of said point cloud data or said plant color data.
 7. Theplant analysis system of claim 6, wherein: said number and size offruits or blossoms on the plant is determined by clustering said pointcloud data and plant color data; and a crop yield is estimated based, atleast in part, on said number and size of fruits or blossoms on theplant.
 8. The plant analysis system of claim 1, wherein said processoris further configured to compare the spectral signature of the plantwith a second spectral signature from a library of plant records.
 9. Theplant analysis system of claim 8, wherein said processor is furtherconfigured to: detect a presence of a plant disease based on saidcomparison of the spectral signature with the second spectral signature;detect a presence of wilt or leaf drop caused by environmental stressorsbased on said comparison of the spectral signature with the secondspectral signature; or identify a pest, including a disease vector, andpredict said pest's trajectory.
 10. The plant analysis system of claim1, wherein a library of plant records is further configured to storehistorical data associated with the plant, said historical datacomprising one or more of date of planting, root stock, grating record,nutritional treatment, health treatment, yield, surrounding soilconditions, surrounding atmospheric conditions, and surroundingtopography.
 11. The plant analysis system of claim 1, further comprisingan asset management dashboard for accessing, viewing, analyzing andcontrolling a library of plant records.
 12. The plant analysis system ofclaim 11, wherein said asset management dashboard integrates spatialinformation of a subset of said plant records in a configurable display,said subset of said plant records correspond to a plot on an orchard,and wherein said asset management dashboard allows users to generatecounting and statistical reports about said plant records.
 13. The plantanalysis system of claim 11, wherein the processor is further configuredto create a record of said plant in a library of plant records, and therecord of said plant associates said plant with said spectral signature,said plant color, said spectral data, said assembled point cloud data,and said morphological data.
 14. A method for analyzing a plant,comprising: measure, using a data acquisition component coupled to atransport vehicle, properties of a plant utilizing waveform lightdetection and ranging (LiDAR) to assemble point cloud data, wherein saidpoint cloud data comprises a plurality of three dimensional verticeseach of which represents an external surface associated with said plant;collect, using the data acquisition component, photographic data of saidplant; collect, using the data acquisition component, spectral data forsaid plant; geo-register the point cloud data and photographic data;associate a plurality of said three dimensional vertices of said pointcloud data with global positioning satellite (GPS) data, saidphotographic data, and said spectral data; determine, using one or moreprocessors, a spectral signature of said plant based on said spectraldata; determine, using the one or more processors, plant color based onsaid photographic data; associate said point cloud data with said plantcolor; and generate, using the one or more processors, morphologicaldata of said plant based on said point cloud data, said morphologicaldata comprising one or more of plant stem diameter, plant height, plantvolume, and plant leaf density.
 15. The method of claim 14, wherein:said morphological data is generated by segmenting said point cloud datato identify boundaries of said plant; or said morphological data isclassified to identify a plant feature, said plant feature comprising abranching structure, trunk, biomass, canopy, fruit, blossom, fruitcluster, or blossom cluster.
 16. The method of claim 14, wherein saidpoint cloud data and said plant color is utilized to discriminate afruit, blossom, fruit cluster, or blossom cluster from a shadow based,at least in part, on analyzing a variation of said plant color and ageometric shape defined by the three dimensional vertices of said pointcloud data.
 17. The method of claim 14, further comprising: measuringatmospheric conditions; and determining a phenotype of the plant basedon said spectral signature, said morphological data, and saidatmospheric conditions.
 18. The method of claim 14, further comprising:determining a number and a size of fruits or blossoms on the plant basedone or more of said point cloud data or said plant color data, whereinsaid number and size of fruits or blossoms on the plant is determined byclustering said point cloud data and plant color data, and a crop yieldis estimated based, at least in part, on said number and size of fruitsor blossoms on the plant.
 19. The method of claim 14, wherein: thespectral signature of the plant is compared with a second spectralsignature from a library of plant records to: detect a presence of aplant disease based on said comparison of the spectral signature withthe second spectral signature; or detect a presence of wilt or leaf dropcaused by environmental stressors based on said comparison of thespectral signature with the second spectral signature.
 20. The method ofclaim 14, further comprising: creating a record of said plant in alibrary of plant records, wherein the record of said plant associatessaid plant with said spectral signature, said plant color, said spectraldata, said assembled point cloud data, and said morphological data.